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FOREWORD 


This  document  frequently  makes  reference  to  the  contents  and  application  of  a  draft  military  standard 
which  is  the  principle  topic  of  this  report.  Although  it  is  not  essential,  it  is  recommended  that  the  reader 
who  is  not  familiar  with  the  Simulator  Data  Integrity  Program  obtain  a  copy  of  MILITARY  STANDARD: 
AIRCREW  TRAINING  EQUIPMENT  SOURCE  DATA  PROCESS  STANDARD.  MIL-STD-XXXX.  OC¬ 
TOBER  1990. 


At  the  time  of  this  writing  MIL-STD-XXXX  is  under  review  within  the  Air  Force.  Requests  for  copies 
should  be  directed  to:  2nd  LL  John  Nehr,  Simulator  Data  Integrity  Program  Program  Manager, 
ASD/YWB  WRIGHT-PATTERSON  AIR  FORCE  BASE,  OH  45433-6503.  TELEPHONE:  DIRECT 
DIAL  513-255-7177,  AUTOVON  785-57177. 


SUMMARY 


The  Simulator  Data  Integrity  Program  is  a  component  of  the  overall  Air  Force  initiatives  to  improve  the 
quality,  performance,  schedules  and  supportability  of  Aircrew  Training  Equipment  This  program 
focuses  on  the  quality,  timeliness,  currency,  and  maintenance  of  the  technical  information  (source  data) 
generated  in  the  weapon  system  organizations  and  used  in  the  training  equipment  development  and  sup¬ 
port  organizations.  The  principle  product  of  this  program  is  the  Military  Standard:  Aircrew  Training 
Equipment  Source-data  Process  Standard  (DRAFT)  MIL-STD-XXXX,  October  1990. 

This  document  is  the  Final  Report  describing  the  development  of  MIL-STD-XXXX  during  the  time  period 
from  January,  1989  through  March,  1991.  The  objective  of  this  report  is  to  generally  describe  the 
program  activities,  findings,  conclusions  and  recommendations  associated  with  the  development  of  MIL- 
STD-XXXX.  This  standard  is  the  product  of  collaboration  with  the  Weapon  System  contractors.  Training 
Equipment  contractors.  Air  Force  and  other  government  organizations  that  participated  in  the 
Industry /Service  Working  Group  (I/SWG)  activities.  The  standard  was  developed  using  the  classic  ap¬ 
proach  of  baselining  the  definitive  end-user  requirements  upon  which  a  ^sterns  engineering  process 
model  was  structured.  The  integrated  requirements  and  the  process  model  went  through  several 
progressive  changes  as  a  result  of  field  surveys  and  subsequent  reviews  by  the  Air  Force  and  the  I/SWG. 

The  adopted  version  of  MIL-STD-XXXX  is  intended  to  be  tailored  for  apphcation  in  the  RFP(s)  for  the 
Major  Weapon  Systems  and  the  associated  Aircrew  Training  Equipment.  The  standard  delineates  the 
requirements  for  a  systems  engineering  approach  to  developing  and  maintaining  source  data  products  at 
the  Weapon  System  and  Training  device  levels.  The  Weapon  System  contractors  will  generally  initiate 
the  processes  by  the  progressive  development  of  training-oriented  source  data  products  that  are  con¬ 
figured  to  the  emerging  Weapon  System  and  the  emerging  Air  Crew  Training  System.  The  processes  in¬ 
clude  Requirements  Analyses,  Product  Development,  Verification,  Integration,  Configuration  Manage¬ 
ment,  Certification,  Validation,  Repositories,  Reporting,  Interface  Control  and  Database  Management 

Similar  processes  are  implemented  by  the  Training  Equipment  contractors  for  the  development  design, 
test  and  Contractor  Logistics  Support  (CLS)  of  the  fidded  training  equipment  Associated  with  this  ef¬ 
fort  is  the  requirement  for  the  Life-cycle  support  of  the  source  data  products  by  all  contractors.  A  key  ob¬ 
jective  of  this  initiative  is  to  minimize  the  time  required  to  acquire  source  data  needed  to  support 
modifications  of  the  training  devices. 


The  principal  conclusions  of  this  report  are: 

0  VALIDATION  OF  PREVIOUS  RESEARCH 

This  effort  has  served  to  re-afhrm  and  eitpand  the  original  findings,  conclusions,  and 
recommendations  of  the  1988  Simulator  Data  Integrity  Study  (ref  ASD-TR-88-5024). 

o  MIL-STD-XXXX  IS  A  VIABI.E  APPROACH 

The  implementation  of  the  Military  Standard:  Aircrew  Training  Equipment  Source  Data 
Process  Standard,  MIL-STD-XXXX;  will  significantly  improve  the  quality,  concurrency 
and  timeliness  of  Aircrew  Training  Equipment 

0  TRAINING  EXPERTISE  UP-FRONT 

The  Weapon  System  contractors,  will  find  it  necessary  to  apply  training  oriented  exper¬ 
tise  to  the  development  of  the  source  data  products  used  by  the  training  equipment 
developers. 
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A  HANDBOOK  IS  NEEDED 

The  initial  application  of  MIL-STD-XXXX  to  any  given  combination  of  emerging  Weapon 
Systems  or  Aircrew  Training  Equipment  requires  the  use  of  a  compamon  reference 
document  (Handbook)  that  provides  guidance  in  the  identification  of  Source  Data  Re¬ 
quirements. 

MIL-STD-XXXX  ENVIRONMENTS 

The  effectiveness  of  MIL-STD-XXXX  will  be  greatly  enhanced  in  the  environment  where 
either  or  both  Concurrent  Engineering  (CE)  and  Computer-aided  Acquisition  and  Logis¬ 
tics  Support  (CALS)  requirements  are  invoked. 

The  principal  recommendations  of  this  report  are: 

0  IMPLEMENTATION  OF  MIL-STD-XXXX 

It  is  recommended  that  the  implementation  of  MIL-STD-XXXX  focus  on  the  objective  of 
integrating  the  requirements  of  the  Simulator  Data  Integrity  Program  within  the  Sys¬ 
tems  Engineering  Master  Schedule  (SEMS)  applied  to  major  weapon  system  acquisi¬ 
tions. 

0  MIL-STD-XXXX  APPLIED  TO  TRAINING  SYSTEMS 

It  is  recommended  that  the  next  revision  of  MIL-STD-XXXX  address  the  requirements 
for  source  data  products  needed  in  all  areas  of  Aircrew  and  Maintenance  Training  Sys¬ 
tems. 

0  EXPLOITING  THE  CALS  INITIATIVES 

It  is  recommended  that  the  application  of  MIL-STD-XXXX  be  structured  to  take  full  ad- 
vantsige  of  the  information  management,  database  requirements  and  capabilities  as¬ 
sociated  with  the  CALS  initiatives. 

0  JOINT  SERVICE  ADAPTATION  OF  MIL-STD-XXXX 

It  is  recommended  that  MIL-STD-XXXX  be  reviewed  and  revised  as  required  for  use  by 
the  U.S.  Navy,  and  the  U.S.  Army  as  a  joint  Military  Standard,  or  a  DOD  Standard. 

0  COLLABORATION  WITH  THE  FLIGHT  TEST  AGENCIES 

It  is  recommended  that  the  appropriate  flight  test  agencies  provide  expertise  and  support 
in  the  development,  verification  and  validation  of  training-critical  MIL-STD-XXXX 
Source  Data  products. 

A  generalixed  description  of  the  Simulator  Data  IntegriQ'’  Program  is  provided  in  Appendix  1,  "Bridging 
the  Information  Gan",  a  paper  presented  at  the  1989 1/lTEC  conference  proceedings. 
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INTRODUCTION: 


The  Simulator  Data  Integrity  Program  is  a  Systems  Engineering  approach  to  the  development  and  main¬ 
tenance  of  the  technical  information  (source  data)  needed  for  the  development,  design,  construction,  in¬ 
tegration,  testing,  maintenance,  modification,  concurrency,  and  life-cycle  support  of  U.  S.  Air  Force, 
Aircrew  Training  Simulators  and  associated  training  equipment.  The  objectives  of  this  program  is  to 
reduce  flight  simulator  development  schedules  and  life  (^cle  costs. 

This  program  is  the  outflow  of  the  Simulator  Data  Integrity  Study,  completed  in  1988,  which  identified 
the  fundamental  problems  associated  with  the  quality,  timehness,  adequacy,  integrity,  interoperability, 
concurrency,  and  management  of  the  source  data  provided  by  the  Weapon  System  organizations  and 
used  by  the  Training  Equipment  and  Training  System  developers  and  operators.  The  final  report  of  that 
effort,  ASD-TR-88-5024,  Simulator  Data  Integrity  Study.  April  1988,  Shaw,  J.J.  and  Gibino,  D.  J.  (DTIC 
AD#  B-131  809)  documents  the  technical  and  non-technical  aspects  of  these  problems.  The  principle 
recommendations  of  the  study  focused  on  the  requirement  for  quality-based  systems  engineering 
processes  for  both  the  weapon  system  contractors  and  the  simulation  equipment  developers  and  users. 

The  purpose  of  this  program  is  to  establish  uniform  practices  that  will  ensure  the  quality,  timeliness,  ef¬ 
fectiveness,  and  supportahility  of  the  vtuious  technical  information  obtained  from  the  weapon  system 
community  and  used  by  the  training  equipment  developers  and  operators.  The  scope  of  effort  is  to  iden¬ 
tify  and  define  the  process  requirements  leading  to  the  development  of  a  military  standard  that  will 
achieve  the  program  objectives.  Associated  with  this  effort  is  the  collaboration  between  industry  and  the 
Air  Force  in  the  form  of  an  Industiy/Service  Working  Group  (I/SWG)  to  provide  feedback  and  develop 
consensus  for  the  process  standard. 

This  report  documents  the  methodology,  findings,  accomplishments,  conclusions,  and  recommendations 
associated  vritb  the  development,  review,  and  introduction  of  the  Military  Standard  -  Aircrew  Training 
Equipment  Source  Data  Process  Standard.  MTT.-STD-XXXX  October,  1990. 

The  structure  of  this  report  is  based  on  the  task  requirements  of  the  Statement  Of  Work,  Project  Plan, 
and  the  completed  effort.  The  report  provides  an  overview  of  the  salient  activities  in  the  four  program 
phases.  It  also  provides  insight  into  the  areas  of  difficully  and  the  recommendations  for  further  con¬ 
sideration.  A  comprehensive  description  of  the  program  is  provided  in  Appendix  1,  "Bridging  the  Infor¬ 
mation  Gap",  a  paper  presented  at  the  1989  Interservice/Industiy  Training  Equipment  Conference. 
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METHODOLOGY: 

The  development  of  the  Aircrew  Training  Equipment  Source  Data  Process  Standard  is  based  on  classic 
Research  and  Development  methods.  The  overall  approach  was  predicated  on  a  detailed  requirements 
definition  that  established  the  baseUne  criteria  for  the  process  standard.  In  parallel  with  the  develop¬ 
ment  of  the  standard  was  the  formative  reviews  by  both  government  and  industiy  organizations.  The 
structured  project  functional  flow  is  illustrated  in  Figure  1  and  described  below; 

Phase  A  -  Project  Planning 

Those  activities  associated  with  the  coordination  and  preparation  of  the  project  plan. 

Phase  B  -  Reauirements/Definition: 

During  this  phase  the  overall  requirements  for  the  source  data  processes  were  identified, 
analyzed  and  integrated  into  a  systems  engineering  oriented  process  model  that  established  the 
process  flow,  relationships  and  interfaces. 

The  research  activities  associated  with  the  identification  of  requirements  was  accomplished 
through  the  use  of  data  searches,  mail  surveys  and  field  surveys.  The  field  surveys  involved  in¬ 
terviews  with  10  Weapon  System  Contractors,  8  Training  System  Contractors,  4  USAF  Using 
Commands,  9  USAF  Aeronautical  Systems  Division  organizations,  3  USAF  Flight  Test  Center 
organizations.  Air  Force  Logistics  Command,  U.S.  Navy  Training  System  Center,  2  Commercial 
Aircraft  Manufacturers,  FAA,  and  others.  It  is  estimated  that  approximately  155  individuals 
participated  in  the  field  surveys  and  interviews. 

The  data  reduction  and  integration  was  accomplished  progressively  and  in  parallel  to  develop  the 
frame  of  reference  for  the  Reqmrements  Taxonomy  and  the  Process  Model.  This  model  evolved 
through  a  series  of  reviews  and  adjustments  to  accommodate  diverse  areas  of  interest.  The  final 
version  of  the  model  is  illustrated  in  Figure  2.  The  integrated  requirements  were  documented  in 
the  Requirements  Taxonomy  provided  at  the  completion  of  Phase  B. 

(NOTE:  The  Requirements  Taxonomy  is  no  longer  current  due  to  the  subsequent 
activities  in  Phases  C  and  D  that  introduced  new,  and/or  revised  requirements  and 
methods.) 

Phase  C  -  Develop  Standard: 

This  phase  consisted  of  two  parallel  efforts  -  preparation  of  the  military  standard  and  the 
Industry /Service  Working  Group  (I/SWG)  review  of  the  military  standard. 

The  preparation  and  authoring  of  the  preliminary  version  of  the  military  standard  was 
based  on  the  approved  Requirements  Taxonomy  and  the  Process  Model  completed  in  the 
preceding  phase  and  further  revised  in  this  phase.  This  version  of  the  standard  was 
reviewed  and  coordinated  with  the  Air  Force  prior  to  release  to  the  I/SWG,  who  par¬ 
ticipated  in  a  series  of  reviews  providing  input  on  the  functionality,  substance,  language 
and  implementation  of  the  standard. 
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The  I/SWG  effort  was  directed  at  obtaining  the  perspectives  and  consensus  of  the  various 
interest  groups  to  the  basic  concept  of  the  processes  under  consideration  for  incorporation 
into  the  standard.  Accordingly,  we  solicited  and  obtained  substantial  verbal  and  written 
input  from  the  I/SWG  representatives.  This  input  was  analyzed  and  reviewed  with  the  Air 
Force  prior  to  integration  into  the  process  standard.  A  synopsis  of  the  overall  response  of 
the  I/SWG  is  included  in  the  Results  section  of  this  report. 

The  final  version  of  the  proposed  draft  Military  Standard,  MIL-STD-XXXX  Aircrew  Train¬ 
ing  Equipment  Source  Data  Process  Standard,  dated  October  1990,  incorporated  the  coor¬ 
dinated  composite  requirements  of  the  I/SWG  and  the  Air  Force  organizations  that  would 
be  responsible  for  the  implementation  of  the  standard.  This  document  was  formally 
delivered  to  the  Air  Force  on  October  4, 1990. 

Phase  D  -  Industry  Review: 

The  final  phase  of  the  effort  to  develop  the  process  standard  was  originally  intended  to 
provide  a  broad-based  industry  review  in  the  form  of  an  Industry/Govemment  Workshop. 
The  Workshop  would  provide  the  means  of  indoctrination  and  familiarization  to  the 
Weapon  System  and  Training  System/Training  Equipment  communities  as  well  as  a 
forum  to  develop  further  improvements  in  MIL-STD-XXXX  and  the  overall  Simulator 
Data  Integrity  Program. 

(NOTE:  The  planning  for  this  phase  mas  changed  to  facilitate  an  addi¬ 
tional  Air  Force  initiative  to  develop  a  Military  Handbook  as  a  companion 
document  to  the  MIL-STD-XXXX.  Accordingly,  it  was  decided  to  defer  the 
broad-based  industry  review  until  the  Handbook  is  sufficiently  developed. ) 


This  approach  was  found  to  be  very  effective  in  that  the  overall  development  of  the  process  standard  was 
a  collaboration  between  diverse  interest  groups.  It  was  essential  to  obtain  and  exchange  insights  into 
needs  and  concerns  of  the  Weapon  System  organizations  and  the  other  organizations  that  are  critically 
dependent  on  the  source  data  generated  in  the  Weapon  System  community. 
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Project  Planning  Functional  Flow 
Figure  1. 


Source  Data  Process  Model 
Figure  2. 


RESULTS: 

The  resijlts  of  this  effort  is  divided  into  two  areas: 


0  Completion  of  the  Military  Standard;  MIL-STD-XXXX. 
0  Completion  of  the  I/SWG  Activities. 


MnJTARY  STANDARD  -  COMPLETION 

The  principal  results  of  this  effort  are  incorporated  in  the  draft  Military  Standard-  Aircrew  Training 
Equipment  Source-Data  Process  Standard.  MIL-STD-XXXX.  October  1990.  It  contains  the  products  of 
the  fact  finding,  investigations,  analyses,  reviews,  I/SWG  feedback  and  other  associated  activities  that 
contribute  to  the  substance  of  the  process  standard.  At  the  time  of  this  writing,  the  October,  1990  Draft 
version  of  MIL-STD-XXXX  is  under  review  within  the  Air  Force,  as  a  preliminary  step  leading  to  the  for¬ 
mal  adoption  and  release  of  the  Military  Standard  in  the  USAF  MIL-PRIME  program. 


I/SWG  ACTIVITIES  -  COMPLETION 

The  secondary  results  of  this  effort  was  the  completion  of  the  I/SWG  related  activities  over  a  period  of 
two  years.  Table  No.2  summarizes  the  I/SWG  representation  during  this  effort. 

The  initial  I/SWG  activities  involved  series  of  field  surveys  and  interviews  with  both  Industry  and 
Government  organizations  to  identify  the  source  data  process  requirements.  Thirty  eight  (38)  organiza¬ 
tions  and  approsimately  155  individuals  participated  in  that  activity. 

The  second  set  of  activities  was  directed  at  a  series  of  three  (3)  I/SWG  meetings  held  in  Manassas,  Vir¬ 
ginia  in  April,  June  and  August,  1990.  These  meetings  provided  valuable  information  from  the 
perspective(s)  of  the  diverse  interest  groups  involved  with  the  development  and  maintenance  of  source 
data.  Generally,  the  information  was  in  the  form  of  verbal  and  written  input  firom  both  Industry  and 
Government  representatives.  The  inputs  were  processed  and  reviewed  by  the  Air  Force  for  integration 
into  MIL-STD-XXXX.  The  objectives  of  the  individual  meetings  and  the  total  I/SWG  effort  were 
achieved.  The  following  is  a  synopsis  of  the  three  meeting  and  associated  activities  based  on  the  meeting 
minutes  and  other  documentation  generated  and  delivered  during  that  time  period: 


A.  PRELIMINARY  I/SWG  ACTiyiTTES: 

The  preliminary  activities  associated  with  the  planning,  preparation,  solicitation  and 
other  management  activities  were  initiated  in  January  1990.  The  principle  elements  of 
this  activity  are; 

1.  The  overall  planning  and  coordination  of  the  I/SWG  effort 

2.  The  preparation  of  the  I/SWG  Charter  and  Frame  of  Reference. 
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3. 


The  preparation  of  the  draft  USAF  letter  to  Industry  that  announced  the  forma¬ 
tion  of  the  I/3WG  and  invited  industry  participation.  The  letter  was  signed  by 
the  ASD  Commander  Lt.  Gen.  Loh,  and  distributed  on  5  March,  1990  to  the 
I/SWG. 

4.  The  liaison  with  Industry  organizations  to  determine  their  interest  and  to  desig¬ 
nate  the  I/SWG  representative(s). 

B.  I/SWG  MEETINGS: 

1.  The  following  tasks  were  completed  for  each  of  the  three  meetings: 

0  preparation  and  coordination  of  all  meeting  materials, 

0  technical  support  to  the  Air  Force  during  the  meetings, 

0  provisioning  of  meeting  facilities  and  equipment,  and 
0  miscellaneous  administrative  support  services. 

2.  I/SWG  Meeting  No.  1  -  (17-19,  April  1990)  Attendance:  60  people  representing 
37  organizations.  This  meeting  served  to  indoctrinate  the  I/SWG  to  the  overall 
program,  familiarization  with  the  source-data  systems  engineering  model  and  to 
review  the  process  standard  requirements  taxonomy.  During  this  meeting  88 
written  inputs  were  received.  The  first  I/SWG  meeting  minutes  (18  May,  1990) 
documents  the  results  of  the  meeting. 

3.  I/SWG  Meeting  No.  2  -  (12-14,  June  1990)  Attendance:  43  people  representing 
34  organizations.  This  meeting  was  directed  at  the  initial  review  of  the  draft 
process  standard,  which  included  a  walk-through  of  the  interim  draft  document 
Each  element  of  the  standard  was  presented  and  discussed  as  appropriate. 
During  this  meeting  126  input  forms  were  received,  and  reviewed  by  the  Air 
Force.  The  meeting  minutes  (11  July,  1990)  documents  the  proceedings. 

4.  I/SWG  Meeting  No.  3  -  (14-16,  August  1990)  Attendance:  34  people  represent¬ 
ing  24  organizations.  This  meeting  addressed  the  revised  draft  process  standard 
and  the  implementation  of  the  Simulatoiy  Data  Integrity  Program.  A  second 
walk-through  of  the  revised  draft  process  standard  was  completed.  The  ap¬ 
proach  to  the  implementation  of  Hie  process  standard  was  discussed  in  con¬ 
siderable  detail.  During  this  meeting  36  input  forms  were  received  and  reviewed 
by  the  Air  Force.  The  meeting  minutes  (  20  Sept.  1990)  documents  the  proceed¬ 
ings  of  this  meeting. 

5.  The  information  obtained  during  the  1/SWG  meetings  was  reviewed  in  detail  by 
SIMTEC  and  the  Air  Force  to  determine  the  requirements  for  changes  to  the 
process  standard.  It  should  be  noted  that  the  majority  input  focused  on  im¬ 
plementation  and  cost  issues  as  opposed  to  the  content  of  the  process  standard. 
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ORGANIZATIONS  SURVEYS  MEETING  MAILLIST 


INDUSTRY  ORGANIZATIONS 
WEAPON  SYSTEM  PRIME  CONTRACTORS 


MAJOR  WEAPON  SYSTEM  CONTRACTORS; 

Boeing  Defense  and  Space  Group 

-  Military  Airplane  Division  x  x 

Douglas  Aircraft  Company  x  x 

General  Dynamics  Corporation 

-  Ft.  Worth  Division  x  x 

Lockheed  Aeronautical  Systems  Company  x 

McDonnell  Aircraft  Company  x  x 

Northrop  Corporation 

-  Aircraft  Division  x  x 

Rockwell  International 

-  North  American  Aircraft  Operations 


PROPULSION  SYSTEM  CONTRACTORS: 

GE  Aircraft  Engines  Engineering  Division  x  x 

Pratt  &  Whitney  Group 

-  Government  Engines  Business  x 

AVIONICS  SYSTEM  CONTRACTORS: 

Honeywell  Defense  Avionics  Systems 
IBM  System  Integration  Division 
Martin-Marietta  Electronics  and  Missiles  Group 
Unisys  Defense  Systems 
Westinghouse  Electric  Corporation 

-  Electronics  Systems  Group 

OTHER  CONTRACTORS: 

Kohlman  Systems  Research,  Inc.  x 

Systems  Control  Technology  (SCT)  x 

TRAINING  SYSTEMS  PRIME  CONTRACTORS 


X  X 

X  X 

X 


X 

X 

X 

X 

X 

X 

X 


X 

X 


X 

X 

X 

X 

X 


X 

X 


TRAINING  EQUIPMENT  CONTRACTORS: 


AAI  Corporation 

X 

X 

X 

Boeing  Aerospace  &  Electronics 

Burtek 

X 

X 

X 

X 

CAE  Electronics  LTD. 

CAE-Link  Corporation 

X 

X 

X 

-  Flight  Simulation  Division 

X 

X 

X 

ECC  International  Corp. 

X 

X 

Flight  Safety  International,  Inc. 

General  Electric  Company 

X 

X 

X 

-  Simulation  &  Control  System  Dept. 

X 

X 

Hughes  Training  and  Support  Systems  Group 

X 

X 

X 

Loral  Defense  Systems-Al^on 

X 

X 

X 

McDonnell  Douglas  Training  Corp. 

X 

X 

X 

Quintron  Corporation 

X 

X 

Reflectone,  Inc. 

X 

X 

OTHER  INDUSTRY  ORGANIZATIONS; 

Boemg  Commercial  Airplane  Group 

X 

X 

X 

Grumman  Electronics  System  Division 

X 

X 

INDUSTRY/GOVERNMENT  WORKING  GROUP  G/SWG)  PARTICIPATION 

Table  1 
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ORGANIZATIONS 


GOVERNMENT  ORGANIZATIONS 

UNITED  STATES  AIR  FORCE: 
AERONAUTICAL  SYSTEMS  DIVISION  (ASO) 
C-17  System  Program  Office  (ASD/YC) 
Advanced  Tactical  Fighter  Program  Office 
(ASD/YF) 

En^eering  (ASD/EN) 

Training  System-System  Program  Office 
(ASD/YW) 

AIR  FORCE  LOGISTICS  COMMAND  (AFLC): 
HiU  AFB  Ogden,  UT  (AFLC  00-ALC/MMI) 

AIR  FORCE  SYSTEMS  COMMAND  (AFSC): 
Air  Force  Flight  Test  Center, 

Edwards  AFB,  CA 

HEADQUARTERS  USAF  (XOOTW) 

TRAINING  COMMAND  (HQ.  ATC) 

MATERIAL  AIR  COMMAND 
(HQ.  MAC/DOT) 

STRATEGIC  AIR  COMMAND 
(HQ.  SAC/XOR) 

TACTICAL  AIR  COMMAND  (TAWC/TNT) 
AIR  NATIONAL  GUARD  (NGB/XOPM) 

U.S.  NAVY 

Naval  Training  Systems  Center 

OTHER  GOVERNMENT  AGENCIES: 

Federal  Aviation  Administration  (FAA) 


SURVEYS  MEETING  MAIL  UST 


X  X 

X  X 

XXX 

XXX 

XXX 

XX  X 

XX  X 

X 

XX  X 

XX  X 

XX  X 

X  X 

XX  X 

XX  X 


INDUSTRY/GOVERNMENT  WORKING  GROUP  (I/SWG)  PARTICIPATION 

Table  1  CondDued 
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I 

FINDINGS; 

The  salient  findings  of  this  effort  are: 

FINDING  No.  1  -  I/SWG  CONSENSUS 

There  is  a  consensus  within  the  organizations  that  participated  in  this  effort,  (see  Table  No.  1) 
that  the  Simulator  Data  Integrity  Program  and  the  fundamental  requirements  of  MIL-STD- 
XXXX  will  yield  significant  improvements  in  the  quality,  effectiveness,  concurrency,  validation, 
and  life-cycle  costs  of  Aircrew  Training  Equipment 


FINDING  No.  2  •  INITIAL  BENEFITS  -  PERFORMANCE  AND  SCHEDULE  IM¬ 
PROVEMENTS 

The  initial  benefits  of  implementing  MIL-STD-XXXX  will  be  evident  in  two  areas:  the  quality  of 
the  end-products  and  the  effectiveness  of  the  training  equipment  test  program.  The  quality  of 
the  training  equipment  will  be  improved  in  direct  proportion  to  the  quality  of  the  design  infor¬ 
mation  provided  to  the  equipment  designers.  Likewise,  the  effectiveness  and  efficiency  of  the 
training  equipment  test  program  will  be  improved  as  a  result  of  the  quality  and  credibility  of  the 
test  information  provided  to  support  the  test  and  evaluatin  processes.  The  combined  effect  of 
these  two  improvements  will  be  increased  training  equipment  performance  and  a  reduced 
program  schedule. 


FINDING  No.  3  •  LONG  TERM  BENEFITS  •  TRAINING  EQUIPMENT  CONCUR¬ 
RENCY 

The  long  term  benefits  of  the  use  of  MIL-STD-XXXX  is  the  life-cycle  availability  of  high  quality 
source  data  products  that  are  maintained  concurrently  with  the  configurations  of  both  the 
Weapon  System  and  Aircrew  Training  System.  The  principal  pay-off  is  the  reduction  in  both 
cost  and  schedule  of  training  equipment  modifications  and  upgrades. 


FINDING  No.  4  •  TIME  RELATIONSHIPS  •  START  EARLY! 

The  effectiveness  and  efficiency  of  the  Simulator  Data  Integrity  Program  will  increase  in  direct 
proportion  to  the  timeliness  of  implementing  MIL-STD-XXXX.  There  are  critical  windows  of  op¬ 
portunity  that  typically  occur  only  once  in  each  Weapon  System  program,  that  must  be  exploited 
to  yield  cost-effective  Training  Simulation  Design  and  Verification  Data.  The  emerging  weapon 
system  programs  responding  to  MIL-STD-XX}6c  starting  at,  or  before.  Milestone  One  will  be 
able  to  take  advantage  of  the  windows  of  opportunity  such  as: 

0  Human  Factors  Engineering  Studies 

0  Mission  and  Task  Analyses 

0  Research  and  Design  (R&D)  Simulations 

-  Propulsion  System  Design 

-  Wind  Tunnel  Studies 

-  Vulnerability  Simulations 

-  Other  Analyses 

o  Test  and  Evaluation  Master  Plan  (TEMP) 

0  Training  System 

-  Analysis  and  Design 

-  Development 

-  Evaluation 
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o 


Weapon  System  Design 

-  Engineering  Development  Simulations 

-  Flight  Control  System  Design 

-  Aerodynamic  Design 

-  Avionics  Design 

-  Failure  Mode  Analyses 

o  Weapon  System  Testing  (DT&E/IOT&E) 

o  Weapon  System  Modification/Update  Programs 

Although  the  processes  delineated  in  MIL-STD-XXXX  are  intended  to  be  applied  to  emerging 
weapon  systems,  it  is  anticipated  that  (with  appropriate  tailoring)  significant  benefits  can  be  real¬ 
ized  in  mature  weapon  systems  undergoing  major  modifications  and  upgrades,  provided  that  the 
effort  is  initiated  at  the  earliest  possible  point  in  time. 


FINDING  No.  5  •  SUPPLEMENTARY  GUIDANCE 

The  identification  of  the  requirements  for  source  data  is  the  most  critical  process  within  MIL- 
STD-XXXX.  The  Weapon  System  contractors  and  the  Training  Eq\aipment  contractors  repre¬ 
sented  in  the  I/SWG  strongly  recommended  that  supplemental  guidance  in  the  form  of  a  Military 
Handbook  should  be  in  place  as  a  companion  document  to  MIL-STD-XXXX.  The  purpose  of  the 
handbook  is  to  provide  a  reference  resource  to  aid  the  weapon  system  and  training  equipment 
contractors  in  the  preliminary  identification  of  the  source  data  needed  for  various  training 
devices.  The  handbook  should  identify  the  essential  characteristics  of  source  data  based  on 
generic  applications.  Appropriate  technical  details  should  be  included  to  permit  the  weapon  sys¬ 
tem  engineers  to  translate  the  generic  characteristics  to  weapon  system  specific  characteristics. 
The  secondary  application  of  the  handbook  is  to  permit  the  weapon  system  contractors  to  es¬ 
timate  the  tasking,  resources  and  schedules  associated  with  the  development  and  life  cycle  sup¬ 
port  of  source  data  products. 


FINDING  No.  6  •  WEAPON  SYSTEM  CONTRACTORS  •  TRAINING  SMART; 

For  the  effective  implementation  of  MIL-STD-XXXX  by  the  Weapon  System  contractors  (primes 
and  subcontractors  alike),  it  is  apparent  that  they  should  apply  training  simulation  oriented  ex¬ 
pertise  and  other  simulation  resources  for  the  development  and  maintenance  of  the  source  data 
products.  These  training  oriented  resources  are  essential  for  the  translation  of  training 
functionality  into  quality  based  source  data  products.  Typically  these  skills  and  resources  must 
be  applied  concurrently  within  the  mainstream  of  the  development  and  testing  of  the  major 
weapon  system  and  it’s  components.  This  is  a  new  role  for  the  Weapon  System  contractors  that 
have  parallels  in  the  design,  manufacture  and  support  of  commercial  airliners. 


FINDING  No.  7  -  AIR  FORCE  IMPLEMENTATION  OF  MIL-STD-XXXX 
MIL-STD-XXXX  was  developed  in  a  fi:ame  work  of  functional  requirements  focused  on  the  tech¬ 
nical  objectives  as  opposed  to  the  contractual  and  acquisition  requirements.  As  the  program 
evolved  the  implementation  issues  began  to  take  form.  During  the  I/SWG  meetings  there  was 
considerable  discussion  by  the  various  interest  groups  (military  and  industry)  as  to  how  to  imple¬ 
ment  the  standard.  These  discussions  covered  a  wide  range  of  implications,  including:  account¬ 
ability,  risk  management,  reporting,  advocacy,  costs,  transfer  of  responsibility,  delivery  and  ac¬ 
ceptance  of  source  data  products,  and  other  related  factors.  It  was  determined  by  the  Air  Force 
that  a  comprehensive  implementation  plan  is  needed  to  effectively  apply  MIL-STD-XXXX  in  any 
given  Weapon  System/Training  System  acquisition. 
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FINDING  No.  8  •  COMMONALITY  CONSIDERATIONS 

There  is  strong  evidence  suggesting  that  the  concepts  of  the  MIL-STD-XXXX  applied  to  other 
areas  of  training  will  also  yield  significant  improvements.  This  was  further  emphasized  by  the 
I/SWG  in  their  recommendations  that  the  process  standard  should  be  expanded  to  include  other 
areas  of  Aircrew  Training  and  Aircraft  Maintenance  Training.  For  example,  in  the  case  of  Main¬ 
tenance  Training  Equipment,  there  is  a  high  level  of  commonality  with  the  source  data  required 
for  Aircrew  Training  Equipment.  Other  potential  applications  include; 

o  Aircrew  Training: 

-  Curriculum 

-  Courseware 

-  Computer  Aided  Instruction  (CAI) 

-  Training  Materials 

0  Maintenance  Training; 

-  Curriculum 

-  Courseware 

-  Computer  Aided  Instruction  (CAI) 

-  Training  Materials 

o  Preparation  of  technical  and  operations  manuals. 

0  Logistics  Support  Analyses. 

FINDING  No.  9  •  ANTICIPATED  PROBLEMS  -  WEAPON  SYSTEM  ORGANIZA¬ 
TIONS 

The  application  of  MEL-STD-XXXX  in  the  weapon  system  community  requires  appropriate 
management  control  to  ensure  effectiveness  in  the  Systems  Engineering  of  the  total  weapon  sys¬ 
tem.  Typically,  there  are  numerous  requirements  for  various  “data"  that  ultimately  compete  for 
resources  in  the  prime  and  subcontractors  engineering  groups.  The  development  of  the  MIL- 
STD-XXXX  Source  Data  Products  requires  the  use  of  the  engineering  resources  from  the  various 
engineering  organizations  that  are  in  the  mainstream  of  the  weapon  system  development  As  es¬ 
tablished  in  the  Commercial  Airline  environment  the  traditional  practices  of  relegating  Training 
Simulator  Data  to  the  ancillary  support  groups  such  as  Ground  Support  Equipment  or  Logistics 
Support  will  not  provide  adequate  results.  Therefore,  the  challenge  to  the  weapon  system  con¬ 
tractors  and  sub-contractors  is  to  provide  the  necessary  management  initiatives  and  program 
priorities  that  will  yield  the  short  and  long  term  benefits  of  the  SDIP. 


FINDING  No.  10- ANTICIPATED  PROBT.EMS  •  TRAINING  ORGANIZATIONS 
The  application  of  MIL-STD-XXXX  by  the  Training  Equipment  contractors  and  sub-contractors 
will  require  special  attention  in  the  areas  of  Source  Data  Product;  Development,  Verification, 
Configuration  Management  and  Validation  processes.  The  requirements  for  these  processes 
must  be  integrated  into  the  training  equipment  Systems  Engineering  disciplines  to  ensure  that 
the  source  data  products  are  maintained  current  with  respect  to  multiple  allocated  baselines. 
The  primary  baseline  is  the  specific  training  equipment  design  criteria.  The  secondary  baselines 
are  the  configuration(s)  of  the  Weapon  System  and  the  configuration(s)  of  the  associated  Aircrew 
Training  System. 

As  in  the  case  of  the  Weapon  System  Contractors  (above),  the  Training  Equipment  Contractor 
and  sub-contractors  will  have  to  break  from  tradition  and  initiate  management  and  engineering 
practices  to  maintain  the  integrity  of  the  source  data,  firom  the  beginning  of  the  development  ef¬ 
fort  to  the  final  delivery  of  the  training  equipment  with  it’s  complement  of  source  data  products. 
In  addition,  the  Contract  Logistics  Support  (CLS)  organization  will  be  responsible  for  the  life 
cycle  continuation  of  the  processes. 
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FINDING  No.  1 1  •  ANTICIPATED  PROBLEMS  -  ACQUISITION  ORGANIZATIONS 
The  overall  requirements  of  MIL-STD-XXXX  are  based  upon  the  current  Air  Force  acqmsition 
strategies  which  includes  the  objective  for  the  contractors  to  accept  full  responsibility  for  the 
deliverable  products.  This  approach  obligates  the  contractors  to  the  development  and  the  evalua¬ 
tion  of  the  source  data  products  prior  to  the  implementation  of  the  products  in  the  development 
and  evaluation  of  the  Aircrew  Training  Equipment.  The  problems  associated  with  this  approach 
is  the  potential  weaknesses  in  quality  assurance  of  the  source  data  products.  This  is  best  il¬ 
lustrated  in  the  case  where  the  government  chooses  to  have  the  training  equipment  developed 
under  a  separate  contract,  i.e.  not  under  the  prime  weapon  system  contract.  In  this  case,  the 
source  data  products  would  pass  from  one  prime  contractor  to  another  with  limited  formal  ac¬ 
ceptance  evaluation  by  the  Government.  In  some  instances  this  approach  will  have  minimal  risk; 
however,  in  some  of  the  more  critical  areas  of  training  simulation  the  risks  are  substantial. 


FINDING  No.  12  -  COMPUTER-AIDED  ACQUISITION  AND  LOGISTICS 
SUPPORT  (CALS) 

In  those  Major  Weapon  System  programs  that  are  designated  as  CALS  candidates,  the  source 
data  products  produced  under  MIL-STD-XXXX  will  be  compatible  with  the  requirements  of  the 
DOD  Computer-Aided  Acquisition  and  Logistics  Support  initiatives  (Reference  MIL-HDBK-59). 
Accordingly,  the  Weapon  System  and  Training  Equipment  Contractors  performing  under  the 
CALS  program  will  be  required  to  incorporate  the  source  data  products  into  the  Contractors  In¬ 
tegrated  Technical  Information  System  (CITIS)  for  direct  access  to  those  with  a  need  to  know. 
This  includes  the  capability  to  transfer  the  source-data  products  via  digital  media  to  the  users. 
This  capability  will  greatly  enhance  the  effectiveness  of  the  overall  process  requirements  of  MIL- 
STD-XXXX. 
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CONCLUSIONS: 


The  fundameatai  conclusions  of  this  effort  are; 

CONCLUSION  No.  1  ■  VALIDATION  OF  PREVIOUS  RESEARCH 

This  program  has  served  to  revalidate  and  expand  the  original  findings,  conclusions, 

and  recommendations  of  the  1988  Simulator  Data  Integrity  Study  (ref  XXX). 

The  members  of  the  I/SWG  and  other  participants  in  this  program  have  confirmed  that  the 
problems  identified  in  the  original  study  have  impacted  programs  and  environments  not  previ¬ 
ously  investigated.  For  example,  in  the  original  study  we  did  not  have  sufficient  information  to 
draw  any  conclusions  with  respect  to  the  situation  where  the  prime  Weapon  System  Contractor 
is  also  responsible  for  the  Aircrew  Training  System.  As  a  result  of  the  current  effort  it  has  been 
determined  that  the  previously  identified  data  problems  are  being  experienced  in  those  programs 
where  the  provider  of  source  data  is  also  the  user.  In  some  instances  the  magnitude  of  the 
problems  were  as  similar  in  scope  as  the  original  findings.  In  one  instance,  this  "in-house" 
problem  caused  significant  program  delays.  In  another  instance,  once  the  "in-house"  problem 
was  identified,  it  took  considerable  time  to  bridge  the  gap  between  the  weapon  system  engineer¬ 
ing  groups  and  the  training  system/training  equipment  developer. 

CONCLUSION  No.  2  •  MIL-STD-XXXX  IS  A  VIABLE  SOLUTION 
The  application  of  the  Military  Standard:  Aircrew  Training  Equipment  Source  Data 
Process  Standard,  MIL-STD-XXXX,  will  significantly  improve  the  quality,  concur¬ 
rency  and  timeliness  of  Aircrew  Training  Equipment. 

There  is  a  consensus  of  the  I/SWG  and  other  interested  groups  that  the  early  application  of  the 
MIL-STD-XXXX  in  the  emerging  weapon  systems,  or  mtyor  updates,  will  yield  both  short  and 
long  term  benefits  to  the  contractors  and  the  Air  Force.  These  benefits  will  be  realized  in  the 
development,  design,  evaluation,  modification,  operation  and  support  of  aircrew  training  equip¬ 
ment.  The  application  of  this  standard  is  characterized  as  an  85%  solution  to  the  traditional 
problems  of  source  data. 

In  three  instances,  major  weapon  system  contractors  have  stated  that  they  will  initiate  the  prin¬ 
cipal  elements  of  this  program  in  advance  of  the  formal  requirement.  In  another  instance  the  Air 
F orce  Logistic  Command  has  indicated  that  they  are  considering  the  application  of  the  standard 
to  an  F- 16  block  modification. 

The  concurrency  of  Aircrew  Training  Equipment  will  be  significantly  improved  by  the  timely 
availability  of  quality  source  data  products  that  will  represent  the  evolving  configurations  of  both 
the  weapon  system  and  the  training  systems.  In  some  instances,  the  I/SWG  has  indicated  that 
50%  of  the  delays  associated  with  simulator  modifications  are  attributed  to  the  lack  of  required 
source  data. 


CONCLUSION  No.  3  -  TRAINING  EXPERTISE  UP-FRONT 

The  Weapon  System  contractors,  will  find  it  necessary  to  apply  training  oriented  ex¬ 
pertise  to  the  development  of  the  source  data  products  used  by  the  training  equip¬ 
ment  developers. 

The  requirements  of  MIL-STD-XXXX  focus  on  the  quality  and  functionality  of  the  source  data 
products  used  in  the  desig;n  and  performance  of  Aircrew  Training  Equipment.  The  development 
of  these  products  requires  expertise  within  the  weapon  system  organizations  that  can  translate 
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the  requirements  for  training  equipment  into  acceptable  source  data  products.  The  training 
oriented  expertise  would  typically  include  training  technologists,  Instructional  System 
Developers,  Training  Simulation  Engineers,  and  other  subject  matter  experts  familiar  with  the 
Air  Force  operating  command  training  methodologies.  This  expertise  would  be  involved  in  the 
following  processes  required  in  MIL-STD-XXXX: 

0  Source-Data  Requirements  Analyses 

0  Source-Data  Product  Development 

0  Source-Data  Product  Validation 

0  Source-Data  Product  Verification 

0  Source-Data  Product  Management 


CONCLUSION  No,  4  - A  HANDBOOK  IS  NEEDED 

The  initial  application  of  MIL-STD-XXXX  to  any  given  combination  of  emerging 
Weapon  Systems  or  Aircrew  Training  Equipment  requires  the  use  of  a  companion 
reference  document  (handbook)  that  provides  guidance  in  the  Source-Data  Require¬ 
ments  Analysis  processes. 

Both  the  Weapon  System  organizations  and  the  Aircrew  Training  Equipment  organizations  in¬ 
volved  in  the  initial  phases  of  development  requires  a  supplementary  resource  or  reference  docu¬ 
ment  that  identifies,  defines,  and  generally  describes  a  generic  set  of  source  data  requirements. 
The  Handbook  delineates  typical  requirements  in  sufBcient  detail  to  permit  the  users  to  address 
the  detail  requirements  for  a  specific  application.  The  Handbook  serves  as  a  tool  or  aid  in  these 
functional  areas: 

0  The  initial  proposal  effort  the  Weapon  System  and  Training  Equipment  contrac¬ 
tors  of  the  scope  of  effort  and  resources  needed  to  develop  the  source  data 
products. 

0  The  initial  Source-Data  Analyses  Processes  in  accordance  with  MIL-STD-XXXX 
paragraph  5.1.1  and  5.2.1. 

(NOTE:  M  the  time  of  this  writing,  the  Air  Force  has  issued  a  contract  to  SIM- 
TEC  Inc.  to  develop  the  Military  Handbook:  Aircrew  Training  Equipment  Source- 
Data  Requirements,  (Draft)  MIL-HDBK-ATESK) 

CONCLUSION  No.  5  ■  MIL-STD-XXXX  ENVIRONMENTS 

The  effectiveness  of  MIL-STD-XXXX  will  be  greatly  enhanced  in  the  environment 
where  either  or  both  Concvurent  Engineering  (CE)  and  Computer-aided  Acquisition 
and  Logistics  Support  (CALS)  requirements  are  invoked. 

The  requirements  of  MIL-STD-XXXX  are  predicated  on  the  principles  of  Systems  Engineering 
applied  to  the  development  and  maintenance  of  Source  Data  Products  concurrently  with  the 
development  of  the  Weapon  System  and  the  associated  Training  Equipment.  Therefore,  as  the 
Weapon  System  development  progresses  the  Training  System  requirements  will  evolve  in  paral¬ 
lel  with  the  Training  Equipment  Source  DAta  Products.  When  this  process  is  started  sufficiently 
early  in  the  weapon  system  program,  the  optimum  level  of  effectiveness  and  efficiency  is  realized. 
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Sustaining  this  process  in  the  out  years  will  maximize  the  return  on  investment  through  concur¬ 
rency  of  the  training  equipment  with  respect  to  the  changing  configurations  of  the  weapon  sys¬ 
tem  and  the  training  system. 

The  application  of  MIL-STD-XXXX  in  those  weapon  systems  programs  that  are  subject  to  the 
CALS  requirements  will  escalate  the  short  and  long  term  benefits  of  the  effort.  The  fundamental 
advantages  of  CALS  complements  the  overall  source-data  processes,  by  the  use  of  integrated  data 
bases  and  digital  transfer  of  the  data  products.  In  the  CALS  environment  both  the  acquisition 
and  implementation  of  source  data  is  more  cost  effective  and  requires  significantly  less  manage¬ 
ment  control. 


RECOMMENDATIONS; 

The  principal  recommendations  are: 


RECOMMENDATION  No,  1  •  IMPLEMENTATION  OF  MIL-STD-XXXX 
It  is  recommended  that  the  implementation  of  MIL-STD-XXXX  focus  on  the  objective 
of  integrating  the  Source  Data  Acquisition  Process  requirements  within  the  Systems 
Engineering  Master  Schedule  (SEIMS)  applied  to  major  weapon  system  acquisitions. 

The  quality  of  training  programs  is  critically  dependent  on  the  timeliness  and  quality  of  the  infor¬ 
mation  obtained  from  the  weapon  system  organizations  for  use  in  the  training  environment.  Ex¬ 
perience  has  shown  that  when  the  weapon  system  contractor(s)  are  properly  motivated,  under 
the  right  conditions,  they  will  provide  effective  source  data.  The  processes  invoked  in  MIL-STD- 
XXXX  are  aligned  with  the  principles  of  Systems  Engineering,  and  the  frame  work  of  the  SEIMS. 
Accordingly,  when  MIL-STD-XXXX  is  invoked  in  the  SEMS  in  the  earliest  phases  of  a  weapon 
system  acquisition  the  maximum  benefits  wiD  be  achieved. 


RECOMMENDATIONS  No.  2  -  MIL-STD-XXXX  APPLIED  TO  TRAINING  SYSTEMS 
It  is  recommended  that  the  next  revision  of  MIL-STD-XXXX  should  encompass  the 
overall  requirements  of  source-data  products  needed  in  all  areas  of  Aircrew  and 
Maintenance  Training  Systems. 

The  current  version  of  MIL-STD-XXXX  focuses  on  the  requirements  of  source  data  to  support 
the  development  of  Aircrew  Training  Equipment  As  identified  by  the  I/SWG  and  others,  the 
issues  and  impact  of  inadequate  low  quality  source  data  have  serious  consequences  in  other  areas 
of  USAF  Aircrew  and  Maintenance  Training  Systems.  Major  shortfalls  in  Instructional  System 
Development  Curriculum  Training  Materials,  Instruction^  Delivery  and  other  training  system 
components  are  attributed  to  deficiencies  in:  source  data  availability,  quality  and  timehness. 
WitMn  these  areas  and  disciplines  there  is  a  high  level  of  commonality  of  source  data  require¬ 
ments. 


(NOTE:  This  recommendation  was  given  serious  consideration  by  the  Air  Force, 
and  in  principle  was  accepted.  Due  to  various  constraints  at  that  time  the  recom¬ 
mendation  was  deferred  far  future  reconsideration.) 


RECOMMENDATION  No.  3  -  EXPLOITING  THE  CALS  INITIATIVES 
It  is  recommended  that  the  application  of  MIL-STD-XXXX  be  structured  to  take  full 
advantage  of  the  information  management  and  database  requirements  and 
capabilities  associated  with  the  CALS  initiatives. 

Contractors  responding  to  the  requirements  of  MIL-STD-XXXX  will  by  necessity  propose  to 
implement  the  program  in  various  databases.  The  DOD  CALS  initiatives  provide  an  engineering 
and  data  management  environment  for  the  overall  weapon  system,  of  which  the  requirements  of 
MIL-STD-XXXX  is  a  very  small  component.  The  integrated  capabilities  of  the  CALS  initiatives 
will  serve  to  reduce  the  cost  of:  development,  configuration  management,  accountability,  data 
management,  data  ownership  and  interface  management  associated  of  the  source  data  products. 
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RECOMMENDATION  No.  4  •  JOINT  SERVICE  ADOPTION  OF  MIL-STD-XXXX 
It  is  recommended  that  MIL-STD-XXXX  be  reviewed  and  revised  as  required  for  use 
by  the  U.S.  Navy  and  the  U.S.  Army  as  a  joint  Military  Standard  or  a  DOD  Standard. 

The  adoption  of  MIL-STD-XXXX  for  use  by  the  Navy  and  the  Army  is  an  essential  step  in  estab¬ 
lishing  the  acceptability,  influence  and  credibility  of  a  uniform  standard.  It  is  clearly  mutual  ad¬ 
vantage  to  all  services  to  ensure  that  this  standard  will  meet  their  unique  requirements  and  to 
conform  to  the  DOD  standardization  programs.  For  example  it  wotild  be  both  cost-effective  and 
practical  to  invoke  a  USAF/Navy  MIL-STD-XXXX  in  the  emerging  acquisition  of  the  Joint 
Primary  Aircraft  Training  System  (JPATS). 


RECOMMENDATION  No.  5  ■  COLLABORATION  WITH  THE  FLIGHT  TEST 
AGENCIES 

It  is  recommended  that  the  appropriate  flight  test  agencies  provide  expertise  and 
support  in  the  development,  verification  and  validation  of  training-critical  MIL- 
STD-XXXX  Source  Data  Products. 

The  MIL-STD-XXXX  processes  for  the  development,  verification  and  validation  of  Source  Data 
Products  are  accomplished  solely  by  the  developing  contractors  and  monitored  by  the  Air  Force. 
This  general  approach  is  considered  sufficient  in  some  areas,  but  marginal  in  areas  that  are 
performance-sensitive  for  training  objectives,  and  the  acceptance  testing  of  the  training  equip¬ 
ment,  The  use  of  the  internal  resources  of  the  Flight,  Avionics,  Propulsion  and  other  Test 
Agencies  to  evaluate  the  source  data  products  will  significantly  improve  the  quality  of  the  source 
data  products  and  simultaneously  reduce  the  attendant  risks.  This  approach  is  effectively  used 
by  the  Navy  in  the  areas  of  Aerodynamics,  Flight  Controls,  and  Handling  Qualities  through  the 
use  of  aircraft  flight  test  resources  from  the  Naval  Aircraft  Test  Center,  Patuxent  River,  MD. 
Similar  approaches  are  used  in  the  Commercial  Airline  environment. 

It  should  be  further  noted  that  the  collaboration  with  the  flight  test  centers  is  mutually  beneficial 
in  that  they  have  a  vested  interest  in  obtaining  quality  source  data  products  for  their  flight  test 
simulation  facilities.  Based  on  previous  investigations  it  was  determined  that  there  is  ap¬ 
proximately  an  80%  commonality  of  requirements  for  source  data  from  the  weapon  system  con¬ 
tractors  for  training  simulation  and  flight  test  simulation  purposes. 
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ABSTRACT 

The  design,  lest,  and  operation  of  aircrew  training  systems  and  aircrew  training  devices  rely  heavily 
upon  source  information  describing  the  weapon  system.  In  the  last  10  years  the  size  of  weapon 
svstem  source  information  has  exploded  into  millions  of  pieces  of  information;  however  management 
approaches  for  acquiring  and  implementing  the  source  information  are  virtually  unchaunged  in  30 
years.  These  traditional  approaches  contain  little  or  no  quality  standards,  are  severely  inadequate  in 
today's  environment,  and  cannot  meet  future  training  needs.  Major  program  problems  stemming 
from  these  inadequacies  include;  extended  development  time,  lack  of  training  program  concurrency 
with  the  weapon  system,  substandard  technical  quality,  higher  program  costs,  and  reduced 
supportability. 

This  paper  presents  an  innovative  approach  to  deahng  with  the  source  information  issue.  The 
Simulator  Data  Integrity  Program,  after  documenting  existing  inadequacies  m  a  1988  research 
report,  is  proceeding  with  the  development  of  a  source  information  process  standard.  This  process 
standard  abandons  the  traditional  approach  of  treating  source  information  as  an  elusive  by-product 
and  instead  treats  it  as  an  integrated  life-cj'cle  process.  When  fully  implemented,  this  new  approach 
will  change  the  way  the  Air  Force,  the  weapon  system  contractors,  and  the  training  system 
contractors  do  business.  It  will  remove  risk  from  training  system  programs  by  putting  the  weapon 
system  source  information  which  drives  training  system  design  and  testing  on  a  much  more  stable 
and  predictable  basis.  This,  in  turn,  will  yield  benefits  in  schedule,  cost,  and  technical  performance. 

This  first  paper  to  industry  presents  the  background  of  the  ongoing  Simulator  Data  Integrity 
Program,  the  present  status  of  the  program,  future  program  objectives  and  the  chaillenges  to  'Doth 
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INTRODUCTION 


The  Information  Gap 

The  development  and  operation  of  aircrew  training 
svstems  and  equipment  is  critically  dependent  on  the 
weapon  system  resources  for  training-oriented 
information  (source  data)  describing  the 
functionality,  operating  characteristics,  environ¬ 
ments,  man-machine  interfaces,  etc.  TS-pically,  the 
training  system  developer/operator  will  solicit  source 
data  from  the  weapon  system  resources,  who  respond 
with  available  data  that  was  generated  for  purposes 
other  than  the  training  of  operator's. 


Figure  1.  Typical  Information  Sources  Required  for  Training 


mainstream  c.'  weapon  system  development.  The" 
mformatiOD  gap  has  serious  consequences:  extensive 
delays,  inordinate  costs  and  protracted  compromises 
m  training  quality. 

Due  to  deficiencies  m  source  data,  traditionally,  the 
training  system  developers  find  it  necessary  to 
empirically  create  and/or  enhance  an  estimated  30% 
of  the  source  data  required  for  recent  weapon  system 
trainers.  In  some  cases  the  weapon  system 
contractor  could  not  provide  the  appropriate  source 
data  because  the  requirement  was  not  known  at  the 
time  the  weapon  system  was  being  developed. 


In  the  case  of  the  USAF  aircrew  training  programs,  to 
provide  proficient:  pilots,  navigators,  engineers, 
electronic  warfare  operators,  refuelers,  mission 
planners,  etc.  for  tomorrow's  weapon  systems,  the 
training  system  developers  and  operators  are 
confronted  with  escalating  requirements  for  source 
data.  In  addition  to  the  weapon  system  technology 
drivers,  is  the  expanding  role  of  aircrew  training 
systems,  to  encompass  coclqiit  resource  management, 
team  training,  mission  rehearsal/planning,  combat 
training  and  Other  training.  These  initiatives  invoke 
requirements  for  improved  quality  and  timeliness  of 
source  data,  in  addition  to  source  data  not  identified 
previously. 

Unfortunately,  the  recent  and  past  performance  in 
the  acquisition  and  implementation  of  source  data 
has  been  substantially  short  of  the  needs.  This 
deficiency  is  characterized  as  the  information  gap 
between  the  weapon  system  contractors  and  the 
training  system  developers/operators. 


What  Kind  of  Information? 

The  source  daU  required  by  the  training  system 
devcloper/operaior  from  the  weapon  system 
resources  varies  as  a  function  of  the  training  program 
objectives.  If  the  training  system  includes 
unsophisticated  training  devices,  the  requirements 
for  data  will  be  substantially  less  than  the  source 
data  required  to  support  a  sophisticated  multi-media 
capability.  Tomorrow’s  training  programs  will 
require  source  data  that  is  more  uniquely  oriented  to 
behavioral  objectives,  man-machine  interfaces, 
complex  environments,  team-training  and  combat 
training  than  is  currently  used  in  today’s  aircrew 
training. 

Figure  1  illustrates  some  examples  of  source  data 
generally  required  in  today’s  training  systems.  This 


The  Information  Gap  has  serious 
consequences:  extensive  delays,  in¬ 
ordinate  costs,  and  protracted 
compromises  in  training  quality. 


The  information  gap  is  not  new,  in  fact  it  has  been  a 
constraint  in  the  military  world  for  as  long  as  training 
has  been  relegated  to  organizations  outside  of  the 


A-4 


Hgiire.does  noi  convey  what,  in  fact,  is  available  or 
applied.  In  some  cases,  this  tvpe  of  data  is  produced 
by  the  weapon  system  contractors  as  part  of  the 
overall  weapon  system  caatractual  data 
requirements.  In  other  cases,  the  data  may  be 
avauable  but  not  formally  produced  for  any  contract 
requirement.  Still  other  data  requires  very  specific 
requirements  definition  and  training-oriented 
development  to  meet  the  needs  of  the  users. 

miere  and  VTien  is  the  Information  Produced? 

The  weapon  system  organizations  that  generate 
source  data  and  the  timing  of  that  activity  are 
essential  to  the  effectiveness,  integrity,  and 
supportability  of  the  information  base  required  for 
training  programs. 

The  origins  of  source  data  are  typically  the  prime 
weapon  system  contractors,  and  the  various  subcon¬ 
tractors  for  engines,  avionics, 
aircraft  subsystems,  electronic 
combat,  etc.  Additionally  there 
are  the  various  organizations  that 
integrate,  test  and  modify  the 
weapon  system.  The  overall 
quality  of  source  data  is  a 
function  of  the  effort  by  the 
weapon  system  developers  to 
provide  information  that  is 
accurate,  consistent,  complete, 
and  appropriate  to  the  need.  If  high-fidelity 
simulation  is  a  training  system  requirement,  then  the 
engineering  groups  within  the  contracting 
organizations  must  provide  definitive  models,  data, 
and  associated  information  to  facilitate  the 
development  and  support  of  the  training  capability. 

In  the  case  of  emerging  weapon  systems,  the 
timeliness  of  source  data  to  facilitate  training  system 
development  is  the  critical  path  for  deploying  the 
required  training  capability  to  support  weapon 
system  readiness.  This  situation  is  compounded  by 
the  unique  window  of  opportunity  for  the 
development,  generation,  integration,  and  validation 
of  source  data  that  is  inextricably  embedded  in  the 
weapon  system  schedule. 

Management  Role 

This  paper  provides  an  overview  of  the  long-term 
problems  associated  with  shortfalls  in  source  data. 
The  deficiencies  in  both  acquisition  and 
implementation  of  source  data  in  the  current  and 
emerging  situation  are  examined  and  the  proposed 
solutions  are  presented.  The  challenges  to  the 
industry  and  the  Government  are  not  constrained  by 
technology,  but  are  instead  rooted  in  the 
management  approach  to  establishing  a  framework 
that  will  motivate  the  weapon  system  contractors, 
training  system  contractors,  and  the  training  system 
operators  to  establish  cost-eflective  uniform  practices 
for  bridging  the  information  gap. 


BACKGROUND 

TraditionHl  .Management  Approaches 

During  ihe  last  10  to  15  years,  the  potential 
capability  of  aircrew  training  systems  and  equipment 
has  increased  dramatically  with  advancements  in 
training  technologies  and  simulation  capabilities. 
Unfortunately,  m  many  cases,  this  potential  is  not 
being  fully  realized.  The  principle  shortfalls  are 
attributed  to  deficiencies  m  quality,  timeliness,  and 
integrity  of  the  source  data  needed  for  the  design  and 
support  of  training  devices.  The  acquisition  of 
training  system/equipment  source  data  has  been 
frequently  imposed  upon  the  weapon  systems 
contractors  through  the  use  of  various  Data  Item 
Descriptions  (DIDs).  For  example,  DI-T-30717 
Simulator  Design,  Data  Requirements  fMay  J977J. 
generally  identifies  generic  data  in  broad  (not 
specific)  terms  and  is  regarded  as  a  tailorabie 
'shopping  list'  from  which 
selections  are  made  in  an  attempt 
to  define  the  training  system  data 
requirements.  Source  data 
acquisition  for  emerging  weapon 
systems  is  typically  initiated  by 
the  Air  Force  in  prime  weapon 
system  acquisition  contracts. 
This  may  be  in  the  form  of  a 
commitment  to  establish  as¬ 
sociate  contracts  with  yet  to  be 
identified  training  system  contractors,  or  alterna¬ 
tively,  the  dau  requirements  DID’s  are  applied  to  the 
contract  during  the  Demonstration/Validation 
(DEM/\'AL)  phase  of  weapon  system  procurement. 
Training  device  manufacturers  are  usually  not  under 
contract  until  1  to  3  years  after  the  weapon  system 
DEM/VAL  Request  for  Proposal  (RFP)  has  been 
released. 

Today,  the  training  system/equipment  developers 
are  responsible  for  the  quality  and  timeliness  of 
source  data  that  was  most  likely  identified  before 
they  were  selected.  They  must  identify  additional 
requirements  and  develop  their  liaisons  and 
contractual  agreements  with  various  weapon  systems 
contractors^  in  hopes  of  obtaining  the  additional 
information.  In  this  scenario,  the  weapon  system 
contractors  may  have  completed  their  design  and 
development  processes  and  be  well  into  DEIM/VAL 
activities  by  the  time  the  training  system  developer 
initiates  bis  request  for  training  system  specific 
source  data.  In  the  case  of  the  existing  weapon 
system,  the  weapon  system  contractor  will  have  long 
since  fulfilled  any  obligation  incurred  by  the  original 
development  processes,  making  the  overall  effort  too 
difficult,  very  costly,  and  much  less  effective. 

Problems  -  Cause  and  Effect 

Mainstream  emphasis  in  training  systems 
development  is  characteristically  aimed  at  fielding  a 

l.  The  number  o(  subcontnetors  in  current  end  emerging  weapon 
lyttem  programs  ranges  from  12  to  35. 


The  challenges. ..are  not  con¬ 
strained  by  technolog>',  but  are 
instead  rooted  in  the  management 
approach. ..to  establishing  cost- 
effective  uniform  practices... 


system  or  device  that  matches  the  current  or 
etncrpng  weapon  system,  as  opposed  to  providing 
continuous  ( long-term /life-cycie)  support  to  account 
for  upgrade?  to  the  weapon  system  or  changes  in 
missions  and  emplovTnent  tactics  aiTectmg  training 
requirements. 

The  prevailing  situation  has  led  to  a  number  of 
quality  problems  impacting  the  effectiveness  of 
training  system/equipment  development  and 
life-cycle  operation. 

Weapon  systems  contractors  are  not,  typically, 
training  oriented  in  their  approach  to  establishing 
criteria  for  source  data  Training  funcuonahty  is 
largely  overlooked  because  there  are  no  definitive 
requirements,  or  guidance  for  the  contractor  to 
address  training  objectives.  This  is  usually 
compounded  by  an  inadequate  training  requirements 
definition  during  program  front-end  analysis. 
Another  contributing  factor  is  the  belief  that  the 
optimal  training  system  configuration  mirrors  the 
weapon  system,  m  the  mistaken  conclusion  that  the 
aircraft  is  the  ideal  learning  tool.  Consequently,  the 
weapon  system  contractor  is  inclined  to  supply 
whatever  information  is  most  readily  available,  and 
not  necessarily  the  data  best  suited  to  meeting 
training  objectives. 

The  shopping  list  approach  of  the  DID  traps  both  the 
weapon  system  and  training  system  houses.  The 
source  data  supplied,  and  accepted,  to  meet  contract 
requirements  are,  in  most  cases,  information  that 
most  closely  relates  to  the  DID  items,  as  opposed  to 
data  determined  to  be  the  most  effective  for  training 
purposes.  In  this  manner,  both  sides  fulfill  their 
contractual  obligations,  but  the  quality  of  the  source 
data  is  not  realized. 

Low-quality,  inadequate,  and  late  source  data  has  led 
to  frustration  in  the  related  engineering  processes 
and  fostered  the  growth  of  practices  based  upon 
heuristic  approaches,  reverse  engineering,  and 
empirically  derived  design  data.  These  approaches 
are  admirable  in  intent  and,  given  the  scope  of  the 
effort,  truly  amazing  in  accomplishment;  however, 
they  lack  the  technical  credibility  and  supportability 
required  for  an  effective  source  data  package.  In 
general,  these  methods  have  not  proved  capable  of 
producing  training  systems/equipment  adaptive  to 
current  demands  for  high  quality  and  concurrency. 

Air  Force  Research 

Increasingly  complex  weapon  systems,  and 
correspondingly  higher  expectations  for  training 
system  performance  are  compelling  the  Air  Force  to 
seek  solutions  to  the  expanding  source  data  shortfalls 
obstructing  the  development  of  training  tools 
essential  for  modem  mission  success.  A  series  of 
initiatives  targeted  toward  achieving  total  quality 
acquisition  of  training  assets  and  life-cycle  operation 
were  established  to  identify  weaknesses  and  promote 


improvements.  One  of  these  initiatives  Was  the 
Simulator  Data  Integrity  Study,  wnich  bad  thfe 
following  goals: 

■  Improie  training  equip  in  enl 

performance. 

a  Reduce  training  equipment 

development  tune. 

•  Reduce  life-cycle  costs. 

•  Provide  timely  updates 
(concurrency). 

•  Reduce  update  costs, 

•  Elxploit  commonality  in  the  use  of 
source  data,  and 

a  Reduce  contractor  bidding  costs. 

The  objectives  of  the  study  were  to  examine  the 
difTiculties  experienced  in  obtammg  comprehensive 
and  timely  source  data,  and  to  propose  changes  to 
assure  that  source  data  requirements  are  satisfied. 
The  research  focused  primarily  on  the  source  data 
required  in  the  development  of  Weapon  Systems 
Trainers  and  Operational  Flight  Trainers  for 
emerging  fixod-wing  tactical  and  transport  aircraft. 

The  study  added  an  exclamation  point  to  the  Air 
Force's  perceptions  of  the  problems.  The  quality  of 
source  data  evidenced  major  deficiencies  especially  in 
microprocessor/soflware  intensive  systems;  Aviomes, 
Electronic  Warfare,  Flight  Controls,  and 
Aerodynamics.  The  source  information  acquisition 
and  implementation  processes  suffered  serious 
shortfalls  due,  in  part,  to  poor  requirements 
definition,  lack  of  relevance  to  training  needs,  and 
lack  of  quality  standards.  These,  and  other  factors, 
combined  to  cause  the  following  consequences; 

•  Delayed  startup  of  simulation  and 
training  programs. 

•  Extended  government  and 
contractor  lest  programs. 

•  Limited  training  effectiveness. 

•  Inordmate  life-cycle  costs. 

•  Excessive  bidding  risks  and 
program  costs. 

•  Unsound  simulator  engineering 
practices. 

The  study  also  highlighted  some  areas  of  successful 
source  data  practices  such  as  the  Air  Force  F-16  Block 
Update  Program,  commercial  airline  and 
FAA-Advanced  Simulation  Programs,  and  the  Naval 
Air  Test  Center  (See  Reference  2).  They  all 
demonstrate  a  capability  for  effective  source  data 
development  and  integration  practices  proving  that 
quality  source  information  can  be  achieved. 

Research  Recommendations 

The  conclusions  led  to  the  study's  recommendations 
for  establishing  a  uniform  process  standard  that  sets 
out  guidelines  for  the  weapon  systems  and  training 
systems  contractors  in  the  acquisition, 
implementation,  and  support  of  source  data. 
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The  Veys  to  this  recommendation  are  the 
management  initiatives  taiten  to  define  training 
source  data  requirements,  ensure  that  correct  data  is 
generated  and  quality  controlled  to  meet  those 
requirements,  and  to  gm^f^tee  that  the  information 
remains  current. 

de\t;loplng  the  solution 


Current  Program 


The  prmciple  recommendation  of  the  Simulator  Data 
Integrity  Study  to,  develop  process  standards  for  the 
acquisition  and  implementation  of  source  data,  is  the 
basis  for  the  current  program,  and  has  this  target 
objective; 

Establish,  implement,  and  sustain  cost-effective 
uniform  practices  that  will  ensure  the  overall  quality 
of  source  data  used  In  the  development,  deployment, 
and  life-cycle  support  of  aircrew  training  systems  and 
associated  equipment. 

To  reach  this  objective,  the  program  is  based  on  a 
structured  approach  that  encompasses  a 
Requirements  Definition  Phase,  a  Process  Standard 
Preparation  Phase,  and  the  Industry  Review  Phase. 
Each  of  these  phases  will  be  supported  by  joint 
industry /USAF  working  groups.  The  Requirements 
Definition  Phase  will  be  completed  in  the  1st  half  of 
1990.  The  Process  Standard  Preparation  Phase  is 
scheduled  for  completion  in  the  3rd  quarter  of  1990, 
followed  by  the  Industry  Review  Phase. 

A  Systems  Enirineei-inr  Management  Anproach 

The  findings  in  the  original  study  establish  that  the 
weaknesses  in  the  acquisition  and  implementation  of 
source  data  are  attributed  to  methodology  not 
technology.  The  current  Air  Force  practices,  which 
are  based  on  traditional  methods,  have  evolved  from 
approaches  used  for  over  30  years.  The  evolution  of 
these  methods  is  seriously  lagging  behind  today's 
needs  and  existing  technologies.  When  considering 
the  implications  of  the  emerging  technologies  and  the 
corresponding  training  requirements  for  the  1990's, 
the  consequences  and  imperatives  are  all  too  clear. 

Preliminary  analysis  of  the  projected  requirements 
for  the  acquisition  and  implementation  of  source  data 
in  the  1990’s  indicates  the  need  for  a  systems 
engineering  approach  that  meets  this  criteria: 


•  A  weapon  system-level  capability 
for  a  training-oriented  source 
data  requirements  analysis  that 
starts  with  inputs  developed  in 
the  concept  definition  phase  of 
the  emerging  weapon  system,  and 
continues  through  the  life  cycle  of 
the  weapon  system. 


conforming  to  the  requirements, 
identified  in  the  aforementioned 
requirements  analysis. 

•  .A  weapon  .system-level  capamliiy 
for  configuration  management, 
quality  control,  repository, 
distribution,  and  interface 
management  of  source  data 
products,  to  be  used  m  various 
training  applications.  This 
capability  shall  be  structured  to 
support  multiple  training  systems 
throughout  the  life  cycle  of  the 
weapon  system. 

•  A  training  system-/training 
equipment-level  capability  for 
source  data  requirements 
analysis,  collection,  and  validation 
of  the  source  data  products 
obtained  from  the  weapon 
system-level  data  base 
(repository).  This  capability  is 
transferred  to  the  operators  and 
support  organizations  after 
deployment  of  the  traimng  assets. 
These  orgamzations  will  continue 
to  use  the  basic  processes 
established  in  the  development  of 
the  training  system. 


•  A  training  system-/training 
equipment-level  capability  for  in¬ 
tegration,  configuration  manage¬ 
ment,  quality  assurance, 
verification,  and  life-cycle  support 
of  source  data  products. 

•  A  capability  within  the  Air  Force 
Test  (Centers  for  the  validation 
and  verification  of  the 
functionality  of  source  data 
products  acquired  by  the  weapon 
system  contractors  and  imple¬ 
mented  by  the  training 
system /training  equipment  cont¬ 
ractors/operators.  In  addition 
the  test  centers  will  provide 
supplementary  source  data  and 
related  expertise. 

•  A  tightly  coupled  interface 
between  all  elements  in  the 
overall  system  that  ensures  these 
source  data  characteristics: 


-  Quality 

-  Timeliness 

-  Concurrency 

-  Supportability 

-  Traceability 


-  Accountability 

-  Commonality 

-  Effectiveness 

-  Efficiency 

-  Integrity 


A  weapon  system-level  capability 
to  develop,  generate,  integrate, 
and  validate  source  data  products 
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Based  on  this  fundamental  criteria  applied  to  a 
variety  of  weapon  systems  (trainer,  fighter,  bomber 


transport,  etc.)  and  varieties  of  training 
systems/ training  equipment,  training  developers/ 
operators,  it  is  apparent  tlial  industry-adopted, 
uniform  practices  are  essential. 

ne\-e)opment  of  Source  Data  Processes 

The  initiai  effort  is  the  determination  of 
requirements  for  the  processes  necessary  to  translate 
functional  needs  into  cost-effective  practices. 

AcQuisition  Processes 

The  proposed  general  processes  for  the  acquisition  of 
source  data  products  (Figure  2)  include  requirements 
analysis,  product  defmition,  development,  validation, 
mtegration,  configuration  management,  data  base 
(repository)  management,  product  catalog,  product 
currency,  quality  management,  and  working  group 
host.  The  weapon  system  contractor  will  initiate 
these  processes  as  part  of  the  demonstration/ 
validation  program  activities,  and  continue 
throughout  the  life  cycle  of  the  weapon  system.  The 
origins  of  source  data  requirements  are  rooted  in  the 
planning,  concept  definitions;  Manpower,  Personnel 
Training  and  Safety.  (MPTSiS);  and  front-end 
analysis  that  precedes  the  weapon  system 
Demonstration/  Validation  phases.  These  processes 
shall  address  the  diverse  requirements  for  source 
data  to  support  a  variety  of  training  applications  and 
multiple  training  systems. 


SOURCE  DATA 

ACQUISITION  PROCESSES 

•  R£OUIR£M£NTS  ANALYSIS 

•  VERIFY  &  Qualify 

•  PRODUCT  definition 

•  maintain  CURRtNCY 

•  DEVELOP  PRODUCTS 

•  quality  MGMT. 

•  VALIDATE  &  INTEGRATE 

•  working  group  MOST 

*  CONFIGURATION  MGMT. 

•  interface  PROCESSES 

•  DATA  base/catalog 

WEAPON  SYSTEM  DEVELOPERS 


Figure  2.  Acquisition  Processes 


Imnlementation  Processes 


Likewise,  the  processes  for  the  implementation  of 
source  data  products  will  be  established  for  the 
training  system/equipment  developers  (Figure  3), 
and  operators.  It  should  be  noted,  that  the  overall 
approach  accommodates  the  potential  for  multiple 
applications  of  source  data  in  various  combinations  of 
training  systems  and/or  training  equipment. 
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throughout  the  life  cycle  of  the  weapon  system.  The 
implementatiCD  processes  for  the  developer  and 
operators  are  essentially  the  same,  except  for  the 
scale  or  volume  of  source  data  products  after  the 
training  systems  are  deployed.  These  processes  will 
encompass  requirements  definition,  source  data 
collection,  validation  of  source  data  products, 
integration,  configuration  management,  data  base 
management,  testing  and  verification,  currency 
management,  quality  control,  and  working  group 
participation  (Figure  4).  The  operator/user  will 
typically  assign  the  implementation  activities  to  the 
logistics  support  components  (depots.  AFLC,  TSSC, 
etc.) 

The  operator's  implementation  of  source  data  is 
driven  by  two  primary  requirements. 

•  Concurrency  to  the  Weapon 
System 

•  Concurrency  to  the  Training 
System 


TRAINING  SYSTEM  NO.  3 
TRAINING  system  no.  2 


TRAINING  SYSTEM  NO.  1 


SOURCE  DATA 

IMPLEMENTATION  PROCESSES 

•  UPDATE  REQUIREMENTS 

•  test/oualify 

•  COLLECT:  NEW/REVISED 

•  maintain  currency 

•  validate 

•  duality  control 

•  integrate 

•  WORKING  GROUP 

•  CONFIGURATION  MGMT 

•  INTERFACE  processes 

j  TRAINING  SYSTEM/EOUIPMENT  OPERATORS 


Figure  4.  Training  System  Operator 
Implementation  Processes 
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Source  Data  Process  Keouirements 

Generally,  the  role  of  the  cognizant  U  S.  .■Ur  Force 
agencies  is  to  establish  the  requirements  i Figure  5) 
and  ensure  the  functionality  of  the  overall  (irocesses. 
The  Air  Force  flight  test  centers  shall  [irovide  various 
levels  of  validation  and  verification  of  the  source  data 
products,  and  in  addition  they  shall  be  a 
supplementary  resource  for  source  data  products. 


SOURCE  DATA 

REQUIREMENTS 

•  WEAPON  SYSTEM 

•  MPTiS/FEA 

PROGHAMS 

•  AIM  FOHCf  TESTING 

•  TRAINI.NC  SYSTEM 

•  WORKING  GROUP 

PROGRAMS 
•  TRAINING  SYSTEM 

■  INTERFACE  PROCESSES 

Planning 

U.S.  Ain  FORCE  AGENCIES 

Figure  5.  Source  Data  Process 
Requirements 


Interface  Processes 

The  definitive  requirements  fdr  interface  processes 
between  weapon  system  developers/operators  and 
the  Air  Force  agencies  have  not  been  determined  at 
this  time.  Obnously,  the  mtcractions  between  the 
users  and  providers  of  source  data  will  be  the  most 
demanding  and  potentially  most  critical.  The 
emerging  UoD  initiatives  for  Computer-aided 
.■\cquisition  and  Logistics  System  tCALS)  shall  be  an 
integral  part  of  the  information  exchange,  and 
interface  processes.  Figure  6  is  a  conceptual  overview 
of  the  broad-based  functional  requirements  for 
processes  to  acquire  and  iinpieineut  source  data 
products. 


COSTS /RENKFITS 

The  fully  integrated  processes  for  the  acquisition  and 
implementation  of  source  data  will  require  up-front 
costs  and  yield  short-  and  long-term  benefits, 
accruing  significant  life-cycle  savings. 

In  the  case  of  emerging  weapon  systems,  the  initial 
costs  are  associated  with  the  detailed  source  data 
requirements  analysis,  and  the  development  of  the 
source  data  products  integral  to  the  mainstream  of 
the  weapon  system  program.  The  cost  efTectivencss 
of  these  efforts  will  be  determined  by  the  plannmg. 
the  integration  of  qualified  resources,  and  the  overall 


Figure  6.  Interface  Processes 
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associated  with  the  day-to-day  support  of  the  data 
base,  repository,  and  user's  interface  will  be  a 
function  of  demand  during  the  weapon  system 
full-scale  development  and  the  level  of  weapon 
system  modifications.  After  the  weapon  svstem 
initial  operational  capability  milestone,  the  level  of 
activity  wdl  be  driven  by  both  weapon  svstem  and 
training  system  changes. 

The  benefits  of  this  capability  are  shared  by  the 
weapon  system  contractors,  the  training  system 
contractors,  and  the  training  system  users.  The 
following  IS  a  summary  of  the  direct  benefits; 

•  Reduction  in  bidding  risks 
associated  with  the  training 
system  procurement  due  to 
availability  of  definitive  source 
data  to  support  program 
development. 

•  Cost  and  schedule  reductions 
attributed  to  significant  decreases 
in  engineering  effort  associated 
wth  the  establishment  of  design 
criteria. 

•  Cost  and  schedule  reductions 
attributed  to  significant  decreases 
in  the  overall  test  and  evaluation 
of  the  training  equipment. 

•  Concurrency  improvements 
resulting  from  improved  response 
times  for  weapon  system  source 
information. 

•  Increased  operational  effective¬ 
ness  attributed  to  the  overall 
quality  of  source  data  used  to 
develop  and  operate  training 
systems. 

•  Increased  suppor lability 
attributed  to  the  overall  integrity, 
accountability,  and  compatibility 
of  source  data  for  the  long-term 
needs  of  training  system 
performance. 

•  Increased  efficiencies  attributed 
to  commonality  of  source  data 
used  in  multiple  applications 
within  and  external  to  the 
training  system.  Integral  to  this 
benefit  are  the  savings  accrued  in 
the  case  of  2nd,  and  3rd 
generation  training  systems  and 
training  equipment  to  support 
the  basic  weapon  system  and  its 
derivatives,  used  by  various  Air 
Force  commands  and  other 
services. 


I  HI-  ^1A^  AI.I  MINI  1  IIAI  I .  r 

The  quality  of  source  available  data  plays  a  crucial 
role  m  management's  ability  to  develop  sirccesslul 
training  systems  on  lime  and  wiihm  budget.  The 
severe  quality  problems  which  exist  due  to  shortfalls 
;n  the  arquisiiion  and  implementation  of  source 
information  have  been  docurueuled  in  itie  Training 
System  Sl’O’s  "Simulator  Uata  lulegritv  Studv  Final 
Kepori"  (Reference  No.l).  Current  practices  of  going 
'oack  to  /Ir  the  data  following  contract  award  for 
weapon  system  full-scuie  development  invariably 
result  m  added  aircrew  training  system  pr-^jram 
costs  and  lost  schedule  time. 

Management  action  is  the  key  to  e.’  niinaling  qualit..v 
information  deficits  resulting  in  cost  and  schedule 
overruns  and  inferior  training  .system  development. 
The  challenges  to  correct  this  debilitating  data 
problem  lie  m  the  ability  and  motivation  of  the 
Government,  the  training  sy.stem  iiidnslry,  and  llie 
weapon  system  manufacturers  to  work  together 
toward  a  soiution.  Here  are  the  challenges: 

•  Vndrr^iandine  thr  Problem  ■  The 
direct  correlation  between  poor 
weapon  system  source  trtforma- 
lion  and  poorly  performing 
training  systems  must  be 
understood  and  acknowledged  by 
the  industry. 

•  Assuming  Responsibility  -  The 
Government,  the  training  system 
industry,  and  the  weapon  system 
contractors  must  each  acknow¬ 
ledge  their  respective  respon¬ 
sibilities.  A  sincere  effort  on  the 
part  of  all  three  is  required  for 
success. 

•  Avoidinp  InaDpropriate  Solutions 
-  Acquiring  source  information  of 
higher  quality  is  a  complex 
technical  and  management  issue. 

It  cannot  be  solved  through  quick 
fixes  or  by  simply  shifting 
contractual  responsibility  without 
first  providing  a  technical 
foundation. 

•  Cooperation  -  Cooperation 
between  the  three  parties 
involved  is  essential  for  success. 

The  challenge  to  management  is 
to  ensure  that  a  positive  ’can-do’ 
atmosphere  exists,  and  to  provide 
the  appropriate  resources  to 
achieve  success. 


The  challenge  to  our  industry. ..is  to 
recognize  that  improvements  in 
training,  program  concurrency, 
cost,  and  technical  performance 
w^ll  accrue  from  this  effort. 
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In  summary,  the  Simuiaior  Data  Integrity  Program 
has,  through  documented  industry  surveys, 
established  that  ineffective  practices  for  source 
information  acquisition  and  iniplenientation  are 
adversely  inijiacting  the  quality  of  aircrew  training 
systems  and  training  dcMces.  These  deficiencies  are 
being  attacked  through  the  development  of  a  source 
data  process  standard  which  accommodates 
technology  projections  and  promotes  a  solution  for 
the  1990’9  and  beyond. 

The  source  data  process  standard  offers  these 
improvements  to  the  training  industry; 

s  A  tailorable  working  tool  clearly 

defining  the  management  and 
technical  processes  to  be  aiiplictl 
during  the  acquisition  and 
implementation  of  source  data. 

•  Processes  providing  for  quality 
and  uniformity  among  the 
multiple  suppliers  of  data  which 
support  a  single  training  system. 

•  A  standard  that  yields  source  data 
products  with  excellent  integrity, 
quality,  and  supportabiUty 
characteristics. 

The  challenge  to  our  industry,  to  Gc-  eminent,  to  the 
weapon  system  contractors  r  1  to  the  training 
system  developers  and  operators,  is  to  recognize  that 
improvements  in  training  program  coucurrency,  cost, 
and  technical  performance  will  acc-ne  from  this 
effort.  We  are  challenged  to  make  a  major  quality 
improvement  in  training  systems,  and  by  working 
sincerely  and  cooperatively  together  our  goal  of 
bridging  the  information  gap  can  be  achieved. 
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